Monitoring web-page content and fetching web-page content may be useful in systems which index or classify such content. For example, search engines, news aggregation services, and other indexing and classification systems may re-visit web-pages from time to time in order to determine whether content associated with those web-pages has changed. Where content has changed, such systems may update indexing and classification data.
Monitoring and fetching systems often visit web-pages in a predetermined fixed order. This approach to monitoring and fetching may be less effective when monitoring highly dynamic web-pages and web-content. For example, visiting web-pages in a predetermined fixed order may be inefficient for monitoring web-pages which are micro-blogs, such as Twitter™.
Thus there exists a need for improved systems and for monitoring content stored at a plurality of locations.
Similar reference numerals are used in different figures to denote similar components.